Feature/add_midas_transformer #1668
Conversation
…series through 'ts_transform()'
…series through 'ts_transform()'
…like they should in case of a MIDAS transformation.
…for every quarter instead of there being some missing months, also changed the docstring a bit such that it works with the debugger
|
@Beerstabr Some tests seem to fail because of |
|
Mmm, okay I will have a look!
Op ma 27 mrt. 2023 om 18:18 schreef madtoinou ***@***.***>
… @Beerstabr <https://github.com/Beerstabr> Some tests seem to fail because
of None values, could you add the necessary logic to the MIDAS
transformation so that these cases are properly handled?
—
Reply to this email directly, view it on GitHub
<#1668 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AZ26ZSPWIV4LCJGPSXAIDQLW6G4U3ANCNFSM6AAAAAAWGJYAUE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@madtoinou it should be fixed now. I wasn't dealing with the 'args' in the correct way, it had changed without me rerunning the tests. |
Codecov ReportPatch coverage:
❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more. Additional details and impacted files@@ Coverage Diff @@
## master #1668 +/- ##
==========================================
- Coverage 94.29% 94.19% -0.10%
==========================================
Files 125 126 +1
Lines 11636 11683 +47
==========================================
+ Hits 10972 11005 +33
- Misses 664 678 +14
☔ View full report in Codecov by Sentry. |
|
Hi @Beerstabr, and thanks for this PR! 🚀 Before I review I was thinking that we could make this an invertible transformer. I would be intersted to see what happens when we have a higher frequency target and lower frequency covariates. What do you think? :) |
|
@dennisbader sounds good! I'll work on it next week! |
|
Hi @Beerstabr , just wanted to check in if there were any updates on this one? :) |
|
Hi @dennisbader, I've been drowned in work lately, and I've not gotten to it. Really would like to finish it though. No updates for now. I have heard this kind of a technique has been implemented in a large retail firm with success (giving a significant increase in accuracy) so it should be worthwhile. |
|
Hi @Beerstabr, thanks for the update. No worries, I totally understand. We can take it over from here. |
|
Hi @Beerstabr, Thank you a lot of the work, I am going to take over this PR and try to finish it in time for the next release 🚀. |
|
@madtoinou thanks! Let me know if something is unclear and I can help to explain it. |
|
Thank you, the code was clear enough. In order to reverse the transformation, the number of component generated during the transform is extracted and used to divide the low frequency period to retrieve the high frequency period. This approach does not work for period above months because of the varying number of days so I added some rule-based logic (pandas Still requires a bit work to properly support multivariate |
|
Closing this as #1820 was merged. Thanks again for this @Beerstabr! 🚀 |
Fixes #1570
Summary
Added a transformer that converts a (high frequency) time series into a lower frequency time series using the mixed-data sampling (MIDAS) approach. For example, covert a monthly time series into a quarterly time series. The benefit of doing this is that you can now use e.g. a monthly time series as an input for a model that forecasts a quarterly targets series without losing (as much) information as you would when you downsample a time series from monthly to quarterly using e.g. pandas.DataFrame.resample.
Limitations:
Possible improvements: